Image coding and decoding method and apparatus considering human visual characteristics

ABSTRACT

An image coding method and apparatus considering human visual characteristics are provided. The image coding method comprises (a) modeling image quality distribution of an input image in units of scenes such that the quality of an image input in units of scenes is gradually lowered from a region of interest to a background region, (b) determining a quantization parameter of each region constituting one scene according to the result of modeling of image quality distribution, (c) quantizing image data in accordance with the quantization parameter, and (d) coding entropy of the quantized image data.

BACKGROUND OF THE INVENTION

This application claims the priority of Korean Patent Application Nos.2002-602, filed on Jan. 5, 2002, and 2002-51883, filed on Aug. 30, 2002,in the Korean Intellectual Property Office, the disclosure of which isincorporated herein in its entirety by reference.

1. Field of the Invention

The present invention relates to image coding, and more particularly, toan image coding and decoding method and apparatus considering humanvisual characteristics.

2. Description of the Related Art

In general, an image is compressed for storage/transmission purposes.FIG. 1 shows a flowchart illustrating a conventional image compressionmethod, wherein, in order to generate a compressed bit stream,spatial/temporal prediction coding (100), transform coding (110),quantization (120), and entropy coding (130), are sequentially carriedout. In this case, most losses are generated during the quantizationoperation 120. This lossy compression method includes a lossycompression method for a still image and a lossy compression method fora moving image. JPEG is a representative lossy compression method for astill image, while MPEG-1, 2, 4, and H.261 and H.263 are representativelossy compression methods for a moving image.

Meanwhile, discrete cosine transform (DCT) is carried out when an imageis coded. In this case, since the amount of calculation is too large inorder to perform DCT on the whole image, an image is divided into blocksof a predetermined size, i.e., 8×8, and is then coded. Also, whenquantization is performed, the amount of information increases if theimage is coded using a quantization parameter for each unit block, so asto make a quantization parameter for each unit block different. Thus,the same quantization parameter is used in the whole image. In MPEG-4and H.263 systems, information is provided in order to adjust aquantization parameter by ±2 for each block of 16×16. Then, theinformation is used to achieve the accurate target bitrate.

When such a coder is used, the image is displayed with similar imagequality on the entire scene. However, when the user looks at an image,the user considers the image quality of a region of interest (ROI) moreimportant than the image quality of a background region. This is whythere is a difference between the regions the user can see at one time.The user intends to look at the region of interest (ROI) more carefullyand to overlook other detailed portions of the background region. Inparticular, this phenomenon remarkably appears in case of a movingimage. Thus, when an image is coded with less bits, improvement of theimage quality of the region of interest (ROI) is needed by allocatingmore bits to the region of interest (ROI) than to the background region,rather than uniformly allocating the bits to the whole image.

In the MPEG-4 and H.263 systems, a part of an image is divided intoregions and coded. In the MPEG-4 system, a user can define the regionsin units of pixels using shape coding beyond a core profile. The abovemethod is mainly used in MPEG-4 because operations can be performed inunits of each object constituting a scene. Each of the objects is codedusing different bitstreams, and user interaction can be performed inMPEG-4 using the above structure. Using this method, the ROI and thebackground region are separated from each other for each object suchthat the image is coded with different image quality. However, thisobject separation process is very complicated. Even though the objectsare simply separated from one another using a rough shape, informationis additionally needed in showing the shape of each of the objects, andthus a compression efficiency is lowered.

Also, in the H.263 system, a part of an image can be divided intoregions in units of groups of consecutive macroblocks (MBs) or in unitsof groups of macroblocks (MBs) in a certain rectangular shape using aslice structured mode at an annex K and the image can be coded. Thismethod, used in the H.263 system, is robust to errors. An importantportion in an environment using a multiple transmission channel istransmitted via a transmission channel in a better environment such thata transmission efficiency is improved and errors occurring in a regionare prevented from spreading into another region. In this case, the ROIcan be coded using a slice structure in a rectangular shape. However, inorder to show the background region, a part of an image must be dividedinto several rectangles, and thus, the structure of the H. 263 systembecomes complicated.

In U.S. Pat. No. 5,764,803 entitled by “Motion-adaptive Modeling SceneContent for very low Bit Rate Model assisted Coding of Video Sequences”,a part of an image is divided into a region of interest (ROI) and abackground region and then, the image is coded. However, there is alimitation in the range of a quantization parameter which can be variedin each region. Thus, due to a difference in the image quality betweenthe region of interest (ROI) and the background region, a boundarybetween the region of interest (ROI) and the background region can beseen.

Also, U.S. Pat. No. 6,263,022 entitled by “System and Method for finegranular scalable (FGS) Video with selective Quality Enhancement”discloses a compression method used in a multiple transmission channelenvironment including a base layer and an enhancement layer. The methodcan adapt to an environment of a transmission channel, but it isdifficult to perform inter prediction, and thus a coding efficiencydecreases. Also, the image quality of the region of interest (ROI) isimproved, but an overall coding efficiency decreases. Thus, the imagequality of the background region is greatly lowered. That is, adifference in image quality between the region of interest and thebackground region increases, and to this end, the boundary between theregion of interest and the background region remarkably appears.

Also, U.S. Pat. No. 6,256,423 entitled by “Intra-frame quantizerSelection for Video Compression” discloses a compression method in whicha region of interest (ROI) and a background region, a transition regionbetween the region of interest (ROI) and the background region aredefined and a quantization parameter between regions is determined. Inthe method, because of the transition region, a phenomenon by which aboundary between the region of interest (ROI) and the background regionappears can be slightly prevented. However, there is a limitation in therange of the quantization parameter of each region, and n transitionregions are also needed when n region of interests (ROIs) exist in apart of a region, and thus a coding method is complicated. In addition,in order to smoothen the boundary between regions, another transitionregion between the transition region and another region is additionallyneeded. As a result, it is difficult to determine a quantizationparameter of each region. In order to solve this problem, a method ofiteratively selecting a quantization parameter has been also used, butthis method results in an increase in the amount of calculation.

SUMMARY OF THE INVENTION

The present invention provides an image coding and decoding method andapparatus, which prevent a boundary between a region of interest (ROI)and a background region from being formed in an image and the methodrequires a small amount of calculation by considering human visualcharacteristics.

The present invention further provides an image coding and decodingmethod and apparatus, by which coding and decoding of a region ofinterest (ROI) are effectively performed using a plurality ofrectangular regions when an image is coded and decoded.

The present invention further provides a recording medium on which theimage coding and decoding method is recorded as a program code that canbe executed by a computer.

According to an aspect of the present invention, there is provided animage coding method. The image coding method comprises (a) modelingimage quality distribution of an input image in units of scenes suchthat the quality of an image input in units of scenes is graduallylowered from a region of interest to a background region, (b)determining a quantization parameter of each region constituting onescene according to the result of modeling of image quality distribution,(c) quantizing image data in accordance with the quantization parameter,and (d) coding entropy of the quantized image data.

According to another aspect of the present invention, there is providedan image coding apparatus. The image coding apparatus includes an imagequality modeling unit which models image quality distribution of aninput image in units of scenes such that the quality of an image inputin units of scenes is gradually lowered from a region of interest to abackground region, and determines a quantization parameter of eachregion constituting one scene according to the result of modeling ofimage quality distribution, an adaptive quantizing unit which quantizesimage data in accordance with the quantization parameter determined bythe image quality modeling unit, and an entropy coding unit which codesentropy of the image data quantized by the adaptive quantizing unit.

According to another aspect of the present invention, there is providedan image decoding method. The image decoding method comprises (a)decoding image data including information on position and size of eachregion and the value of a quantization parameter in a receivedbitstream, (b) determining the value of a quantization parameter in eachregion using the information on position and size and each region andthe value of a quantization parameter restored in (a) such that imagequality is gradually lowered from a region of interest to a backgroundregion, (c) inverse-quantizing decoded image data using the value of thequantization parameter, and (d) adding an image restored for each regionaccording to its corresponding position in accordance with the value ofposition of each region restored in (a) and constituting one scene.

According to another aspect of the present invention, there is providedan image decoding apparatus. The image decoding apparatus includes anentropy decoding unit which decodes entropy of image data includinginformation on position and size of each region and the value of aquantization parameter in a received bitstream, an image qualitymodeling unit which determines the value of a quantization parameter ineach region using the information on-position and size and each regionand the value of a quantization parameter restored by the entropydecoding unit such that image quality is gradually lowered from a regionof interest to a background region, an adaptive inverse-quantizing unitwhich inverse-quantizes the image data provided by the entropy decodingunit according to the value of the quantization parameter for eachregion determined by the image quality modeling unit, and an imageconstituting unit which adds an image restored for each region accordingto its corresponding position in accordance with the value of positionof each region provided by the entropy decoding unit and constitutes onescene.

According to another aspect of the present invention, there is providedan image coding apparatus. The image coding apparatus includes a slicemodeling unit which divides each image into at least one of independentrectangular slices, a picture header coding unit which codes informationon positions and sizes of the slices divided by the slice modeling unitto a picture header together with other information, and a slice codingunit which codes an image in units of slices by referring to the pictureheader information.

According to another aspect of the present invention, there is providedan image decoding apparatus. The image decoding apparatus includes apicture header decoding unit which decodes a picture header in abitstream, a slice constituting unit which constitutes slices usinginformation on positions and sizes of slices among the picture headerinformation, a slice decoding unit which decodes an image in units ofslices by referring to the picture header, and an image constitutingunit which constitutes the image in units of slices restored by theslice decoding unit as a part of an image by referring to theinformation on positions and sizes of the slices obtained by the sliceconstituting unit.

According to another aspect of the present invention, there is providedan image coding method. The image coding method comprises (a) settingpositions and sizes of slices in a part of an image, (b) codinginformation on positions and sizes of the slices set in (a) to a pictureheader together with other information, and (c) coding an image in unitsof slices by referring to the picture header information coded in (b).

According to another aspect of the present invention, there is providedan image decoding method. The image decoding method comprises (a)decoding-a picture header in a bitstream, (b) constituting slices usinginformation on positions and sizes of slices included the picture headerdecoded in (a), (c) decoding an image in units of slices by referring tothe picture header decoded in (a), and (d) constituting the image inunits of slices decoded in (c) as a part of an image by referring to theinformation on positions and sizes of the slices.

According to another aspect of the present invention, there is providedan image coding method in which an image is divided in units of sliceshaving a predetermined size and is coded. The image coding methodcomprises defining information on positions and sizes of a plurality ofrectangular regions in which a region of interest is included in theimage and a larger rectangle includes a smaller rectangle, coding allslices included in a smallest inner rectangular region among theplurality of rectangular regions, coding slices in which outerrectangles excluding the inner rectangular region are not overlapped onsmaller inner rectangles, and defining a region not included in anoutermost outer rectangular region as a background region and codingslices included in the background region.

According to another aspect of the present invention, there is providedan image decoding method in which an image is divided in units of sliceshaving a predetermined size and is coded, the image is divided into abackground region and a region of interest, and the image is decoded ina bitstream defined by a plurality of rectangular regions in which aregion of interest is included in the image and a larger rectangleincludes a smaller rectangle. The image decoding method comprisesextracting information on each position and size of the plurality ofrectangular regions including the region of interest from the bitstream,decoding all slices included in a smallest inner rectangular regionamong the plurality of rectangular regions, decoding only slices inwhich outer rectangles excluding the inner rectangular region are notoverlapped on smaller inner rectangles, and decoding all slices in thebackground region not included in the plurality of rectangular regions.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other features and advantages of the present inventionwill become more apparent by describing in detail preferred embodimentsthereof with reference to the attached drawings in which:

FIG. 1 shows a flowchart illustrating a conventional image compressionmethod;

FIG. 2 shows a block diagram schematically illustrating an image codingapparatus considering human visual characteristics according to anembodiment of the present invention;

FIG. 3 shows a flowchart illustrating an image coding method performedin the image coding apparatus of FIG. 2, according to an embodiment ofthe present invention;

FIGS. 4A and 4B show a step of determining a region of interest (ROI),an interpolation region, and a background region in one scene;

FIGS. 5A through 5C show graphs comparing the distribution of imagequality of the prior art with distribution of image quality of thepresent invention;

FIGS. 6A through 6C show the characteristics of a quantization parameterdetermined by an image quality modeling unit of FIG. 2;

FIGS. 7A and 7B show the number of bits versus image quality inaccordance with variations of a quantization parameter in a linear ornonlinear quantization method;

FIGS. 8A through 8C show comparison of a case where an image codingmethod according to the present invention is applied to an actual image,with a case where a conventional coding method is applied to an actualimage;

FIG. 9 shows a block diagram schematically illustrating an imagedecoding apparatus considering human visual characteristics according toan embodiment of the present invention;

FIG. 10 shows a flowchart illustrating an image decoding methodperformed in the image decoding apparatus of FIG. 8, according to anembodiment of the present invention;

FIG. 11 shows a block diagram of an image coding apparatus according toan embodiment of the present invention;

FIG. 12 shows a flowchart illustrating an image coding method performedin the image coding apparatus of FIG. 11;

FIG. 13 shows a block diagram schematically illustrating an imagedecoding apparatus according to another embodiment of the presentinvention;

FIG. 14 shows a flowchart illustrating an image decoding methodperformed in the image decoding apparatus of FIG. 13;

FIGS. 15A and 15B illustrate a method of processing slices according tothe present invention;

FIG. 16 shows comparison of a subjective image quality of a case where acoding method according to the present invention is applied to an actualimage, with a case where a conventional coding method is applied to anactual image; and

FIG. 17 shows comparison of an objective image quality of a case where acoding method according to the present invention is applied to an actualimage, with a case where a conventional coding method is applied to anactual image.

DETAILED DESCRIPTION OF THE INVENTION

Hereinafter, an image coding and decoding method and apparatusconsidering human visual characteristics according to the presentinvention will be described in detail with reference to the accompanyingdrawings.

FIG. 2 shows a block diagram schematically illustrating an image codingapparatus considering human visual characteristics according to anembodiment of the present invention. Referring to FIG. 2, the imagecoding apparatus includes an image quality modeling unit 140, aprediction coding unit 145, a transform coding unit 150, an adaptivequantizing unit 155, and an entropy coding unit 160.

The image quality modeling unit 140 models image quality distribution ofan input image in units of scenes such that the image quality of animage input in units of scenes is gradually lowered from a region ofinterest (ROI) to a background region. Also, the image quality modelingunit 140 determines a quantization parameter of each of regionsconstituting one scene in accordance with the result of modeling ofimage quality distribution.

The predication coding unit 145 prediction-codes the input image, andthe transform coding unit 150 transform-codes the prediction-coded inputimage. In this case, in order to simplify coding calculation, thepredication coding unit 145 and the transform coding unit 150 performcoding in units of blocks having a predetermined size.

The adaptive quantizing unit 155 quantizes image data for each MB inaccordance with the quantization parameter determined by the imagequality modeling unit 140.

The entropy coding unit 160 codes entropy of the image data quantized bythe adaptive quantizing unit 155.

FIG. 3 shows a flowchart illustrating an image coding method performedin the image coding apparatus of FIG. 2, according to an embodiment ofthe present invention.

Now, the operation of the image coding apparatus of FIG. 2 will bedescribed with reference to FIGS. 2 and 3.

First, in step 200, the image quality modeling unit 140 models imagequality distribution on an input image in units of scenes such that theimage quality of an image input in units of scenes is gradually loweredfrom a region of interest (ROI) to a background region. After step 200,in step 210, the image quality modeling unit 140 determines aquantization parameter of each of blocks having a predetermined size,i.e., 8 (pixel)×8 or 16×16; constituting one scene in accordance withthe result of modeling of the image quality distribution and providesthe determined quantization parameter to the adaptive quantizing unit155.

Since the image quality modeling unit 140 models image quality such thatthe image quality of the input image is gradually lowered from a centerof the region of interest (ROI) to the background region, thequantization parameter is small in the region of interest (ROI) andgradually increases toward the background region. The quantizationparameter is determined by a bit target. Here, the image qualitymodeling unit 140 can model the image quality so that image qualitydistribution has gaussian distribution. Alternatively, the image qualitymodeling unit 140 can set a predetermined interpolation region betweenthe region of interest (ROI) and the background region for each sceneand can model the image quality distribution such that the highest imagequality is obtained in the center of the region of interest (ROI) andimage quality for each region is lowered in the background region, but adifference in the background region is not remarkable in theinterpolation region. For example, the image quality modeling unit 140can determine a region of interest (ROI) 250 and a background region 260in one scene, as shown in FIG. 4A, and set an interpolation region 300for removing a blocking phenomenon caused by a difference in imagequality between regions in a region in which each of the regions isconnected to each other, as shown in FIG. 4B. The image quality modelingunit 140 determines a quantization parameter by modeling image qualitysuch that image quality is naturally connected in the interpolationregion. A quantization parameter in the interpolation region can bevaried linearly or nonlinearly. Modeling of image quality will bedescribed in detail with reference to FIGS. 5 and 6.

After step 210, in step 220, the adaptive quantizing unit 155 inputsprediction-coded and transform-coded image data for each region, and theimage quality modeling unit 140 quantizes the image data for each regionin accordance with the quantization parameter determined by the imagequality modeling unit 140. Meanwhile, the quantization parameterprovided by the image quality modeling unit 140 increases from theregion of interest (ROI) to the background region of a scene, and thusthe degree of losses caused by quantization is different. That is, theleast loss is in the center of the region of interest (ROI) at which thehuman eyes are most focused, and losses increase toward the backgroundregion at which the human eyes are relatively least focused.

After step 220, in step 230, the entropy coding unit 160 codes entropyof the image data quantized by the adaptive quantizing unit 155 andoutputs the coded entropy of the image data as a bitstream.

As above, the quantization parameter gradually increases from the regionof interest (ROI) to the background region through predeterminedimage-quality modeling considering human visual characteristics, therebyeffectively removing the blocking phenomenon between regions whileimproving a coding efficiency.

FIGS. 5A through 5C show graphs comparing the distribution of imagequality of the prior art with distribution of image quality of thepresent invention. In FIGS. 5A through 5C, an x-axis and a y-axisindicate a spatial domain of each image, respectively, and a z-axisindicates image quality. And, a region of interest (ROI) of an image isassumed to be a middle portion of a scene.

FIG. 5A shows image quality distribution illustrating a case where theimage quality modeling unit 140 models image quality distribution in onescene to have gaussian distribution centered on the region of interest(ROI). Referring to FIG. 5A, image quality distribution hastwo-dimensional gaussian distribution averaged on the center of animage, in which the highest image quality is obtained in the region ofinterest (ROI) and image quality is gradually lowered in the backgroundregion. In this case, the slope of variations in image quality can beadjusted in accordance with dispersion values of the x- and y-axes. Theimage quality distribution can be formed differently in accordance withthe number of regions of interest (ROI). In addition, in this case,quantization parameter distribution caused by image quality distributionis modeled in one scene and a model thereof is transmitted, instead ofcoding a quantization parameter for each unit block and transmitting thequantization parameter. For example, if the position and dispersionvalue of the unit block corresponding to the center of the region ofinterest (ROI) are transmitted, the quantization parameter of all unitblocks can be calculated automatically at a recipient.

FIG. 5B shows a case where the interpolation region is placed betweenthe region of interest (ROI) and the background region, so as to reducea rapid difference in image quality. In this case, one quantizationparameter is assigned to each region. An interval at which aquantization parameter is to be varied, is set in accordance with thesize of a region. Additional information does not need to be inserted inthe interval by setting the same rules in a coder and a decoder. Forexample, in the case of a rectangular region, if a region correspondingto 20% of each of width and length is set to an interval at which thequantization parameter is to be varied, the varied quantizationparameter can be applied to the region of a block corresponding to 20%from the outside of each region and can be coded. Even when decoding thequantization parameter, the varied quantization parameter is applied toa portion corresponding to 20%, and thus the varied quantizationparameter can be correctly coded. And, by using the same quantizationmethod in the coder and the decoder, the quantization parameter can bevaried in according to preset rules without inserting additionalinformation on a method for varying the quantization parameter.

FIG. 5C shows image quality distribution illustrating a case where onescene is divided into a region of interest (ROI) and a background regionin the prior art. In this case, due to a difference in image qualitybetween two regions, a blocking phenomenon that a boundary between thetwo regions appears, occurs and thus image quality is damaged.

Finally, referring to FIGS. 5A and 5B, image quality is graduallylowered from the region of interest (ROI) to the background region, andthus a user cannot sense a difference in image quality between theregion of interest (ROI) and the background region. However, due to arapid difference in image quality between the region of interest (ROI)and the background region, as shown in FIG. 5C, the user senses theblocking phenomenon.

FIGS. 6A through 6C show the characteristics of a quantization parameterdetermined by the image quality modeling unit 140 of FIG. 2. Forconvenience, assuming that 0-2 and 8-10 of a spatial domain are set tothe background region, 2-8 of the spatial domain is set to the region ofinterest (ROI), and the quantization parameter is varied from 0 to 31.

FIG. 6A shows a case where the quantization parameter is linearly variedin the interpolation region of FIG. 4B or 5B, and FIG. 6B shows a casewhere the quantization parameter is nonlinearly varied in theinterpolation region of FIG. 4B or 5B. Referring to FIGS. 6A and 6B, thequantization parameter is determined such that a difference in imagequality between the region of interest (ROI) and the background regionis not rapidly varied but is gradually varied in the interpolationregion between the region of interest (ROI) and the background region.FIG. 6C shows the characteristics of the quantization parameter when onescene is divided into the region of interest (ROI) and the backgroundregion without a conventional interpolation region, and in FIG. 6C, adifference in the quantization parameter between the region of interest(ROI) and the background region is rapid. As such, the blockingphenomenon may occur at the boundary between the region of interest(ROI) and the background region.

Meanwhile, the linear/nonlinear characteristics of the quantizationparameter can be determined by the characteristics of a quantizationmethod, as shown in FIGS. 6A and 6B. That is, whether the number of bitsand image quality are varied linearly or nonlinearly, is determined inaccordance with variations in the quantization parameter.

FIGS. 7A and 7B show the number of bits versus image quality inaccordance with variations of a quantization parameter in a linear ornonlinear quantization method, and MPEG-4 and H.26L coders are used inFIGS. 7A and 7B.

FIG. 7A shows variations in image quality in accordance with variationsof a quantization parameter and shows that the MPEG-4 coder hasnonlinear characteristics and the H.26L coder has linearcharacteristics. In addition, FIG. 7B shows variations in the number ofbits according to variations of the quantization parameter and showsthat the MPEG-4 coder has nonlinear characteristics and the H.26L coderhas linear characteristics. That is, a method for varying thequantization parameter linearly or nonlinearly is determined by aquantization method, and thus transmission of additional information isnot needed. Also, the number of bits is reduced in the region ofinterest (ROI), and the number of bits is increased toward thebackground region. Thus, the entire number of bits is not largely variedand additional calculation is not needed.

FIGS. 8A through 8C show comparison of a case where a coding methodaccording to the present invention is applied to an actual image, with acase where a conventional coding method is applied to an actual image.FIG. 8A shows a case where the adaptive quantization method according tothe present invention is applied, and FIG. 8B shows a case where arectangular region of interest (ROI) is formed in a middle portion of animage, the region of a scene is divided into two regions, and differentquantization parameters are coded in each region. Also, FIG. 8C shows acase where the same quantization parameter is coded in all blocks of ascene without dividing the region of interest (ROI).

Referring to FIGS. 8A through 8C, the subjective image quality of animage can be improved using coding using the region of interest (ROI),as shown in FIG. 8B, rather than using the same quantization parameterin all blocks, as shown in FIG. 8C. However, a boundary between the ROIand the background region is formed. Due to this boundary phenomenon,there is a limitation in making a difference in image quality betweenthe ROI and the background region such that there is a limitation in anefficiency of coding the ROI. However, in the case of the adaptivecoding method as shown in FIG. 8A, an interpolation region is setbetween the ROI and the background region, and image quality isgradually varied in the interpolation region such that a difference inimage quality between two interpolation regions does not appearremarkably.

FIG. 9 shows a block diagram schematically illustrating an imagedecoding apparatus considering human visual characteristics according toan embodiment of the present invention. Referring to FIG. 9, the imagedecoding apparatus includes an image quality modeling unit 300, anentropy decoding unit 310, an adaptive inverse-quantizing unit 320, animage restoring unit 330, and an image constituting unit 340.

FIG. 10 shows a flowchart illustrating an image decoding methodperformed in the image decoding apparatus of FIG. 8, according to anembodiment of the present invention. Referring to FIGS. 9 and 10, theentropy decoding unit 310 receives a bitstream and decodes the bitstreamof data belonging to each of regions. In step 410, image data includinginformation on positions of each of the regions, the value of aquantization parameter, and the sizes of each of the region are decodedin the bitstream.

In step 420, the image quality modeling unit 300 determines the value ofa quantization parameter of a corresponding region using the informationon positions and sizes of each of the regions and the value of aquantization parameter of the data decoded by the entropy decoding unit310. The image quality modeling unit 300 provides the value of thequantization parameter to the adaptive quantizing unit 320. In thiscase, the value of the quantization parameter may be set to be graduallyvaried at a predetermined interval so that the value of the quantizationparameter between each of the regions is not rapidly varied. Asdescribed above with reference to FIGS. 5A through 5C and 6A through 6C,image quality distribution may be modeled to have gaussian distributioncentered on a region of interest (ROI), or an interpolation region maybe placed between the ROI and a background region so as to reduced arapid difference in image quality. In this way, setting an intervalwherein the value of quantization parameter for each region determinedby the image quality modeling unit 300 is to be varied, or varying aquantization parameter is performed as in the above-mentioned codingmethod.

In step 430, the adaptive inverse-quantizing unit 320 inverse-quantizesdata for each block transmitted by the entropy decoding unit 310 usingthe value of the quantization parameter determined by the image qualitymodeling unit 300.

In step 440, the image restoring unit 330 performs inverse-transform oneach block having a predetermined size, compensates predictedinformation, and restores an image.

In step 450, the image constituting unit 340 constitutes one scene byadding the restored image for each region according to its correspondingposition in accordance with the information on positions of each of theregions provided by the entropy decoding unit 310, to a part of animage.

The structure and operation of the adaptive image decoding apparatusaccording to the present invention described above are used to decode animage coded by the adaptive image coding apparatus of FIG. 2. Theoperation is performed in a reverse order to that the adaptive imagecoding apparatus, and setting an interval wherein the value of aquantization parameter for each block is to be varied, or varying aquanitzer parameter performed in the adaptive image decoding apparatusis performed as in the coding method. Thus, for convenience, theseoperations will not be repeated here.

FIG. 11 shows a block diagram of an image coding apparatus according toanother embodiment of the present invention. Referring to FIG. 11, theimage coding apparatus according to the present invention includes aslice modeling unit 1100, a picture header coding unit 1200, and a slicecoding unit 1300. The slice coding unit 1300 includes a spatial/temporalprediction coding portion 1310, a transform and quantizing portion 1320,and an entropy coding portion 1330.

The slice modeling unit 1100 divides an image 1000 into at least one ofindependent slices so as to independently code an arbitrary regiondesired by a user. That is, the slice modeling unit 1100 may define theROI as a rectangular region and may constitute the ROI and a regionoutside the ROI of a plurality of independent slices. In addition, in apart of the image 1000, the slice modeling unit 1100 may constitute aregion so that a large rectangular region is overlapped on a smallrectangular region using several rectangular regions and may constitutea small rectangular region and a large rectangular region that is notoverlapped on the small rectangular region of a plurality of independentslices.

The picture header coding unit 1200 codes common information needed indecoding all slices in an image and transmits the coded information tothe slice coding unit 1300. In this case, the number, shape, position,and size of slices are included in the transmitted information.

The slice coding unit 1300 codes the image in units of slices byreferring to picture header information input from the picture headercoding unit 1200. For this purpose, the spatial/temporal predictioncoding portion 1310 removes spatially and temporally overlappedinformation. The transform and quantizing portion 1320 performspredetermined transform, for example, DCT, on an output of thespatial/temporal prediction coding portion 1310 and quantizes atransform parameter. The entropy coding portion 1330 codes entropy of anoutput of the transform and quantizing portion 1320 and generates acompressed bitstream.

The slice coding unit 1300 divides an image in units of slices when theimage is coded and transmitted via a network. In particular, the imageis divided in units of rectangular slices and coded, and thus, aninter-slice prediction loss is reduced. Also, ROI coding andpicture-in-picture (PIP) coding are performed using a slice structuredivided into rectangular inner and outer regions. Here, in the ROIcoding, the image is divided into a region of interest (ROI) and abackground region, and image quality of the ROI is increased, and imagequality of the background region is lowered such that subjective imagequality is improved using restricted bit rate. In the PIP coding, aportion comprised of rectangular slices can be independently decodedsuch that the portion is used like another image.

According to the present invention, when a rectangular region comprisedof a plurality of slices is overlapped on another rectangular region, apredetermined region between the overlapped portion and the backgroundregion is set as an interpolation region such that during the ROI codingoperation, due to a rapid variation of image quality between the ROI andthe background region, subjective image quality is prevented from beinglowered. In addition, during the PIP coding operation, PIP havingvarious sizes can be used.

FIG. 12 shows a flowchart illustrating an image coding method performedin the image coding apparatus of FIG. 11. Referring to FIG. 12, ifposition and size of a rectangular region to be independently processedare set, in step 1400, the slice modeling unit 1100 divides thecorresponding rectangular region into at least one of independentslices. In step 1500, the picture header coding unit 1200 codes apicture header, and in step 1600, coding on slices is performed by theslice coding unit 1300.

Here, slice coding (step 1600) comprises spatial/temporal predictioncoding of removing spatially and temporally overlapped informationexisting in an image in units of slices (step 1610), performingpredetermined transform, for example, DCT, and quantization on the datafrom which the overlapped information is removed (step 1620), and codingentropy of the quantized data and generating a compressed bitstream(step 1630).

FIG. 13 shows a block diagram schematically illustrating an imagedecoding apparatus according to another embodiment of the presentinvention. Referring to FIG. 13, the image decoding apparatus accordingto the present invention includes a picture header decoding unit 2000, aslice constituting unit 2100, a slice decoding unit 2200, and an imageconstituting unit 2300. The slice decoding unit 2200 includes an entropydecoding portion 2210, an inverse-quantizing and inverse-transformportion 2220, and an image restoring portion 2230.

The picture header decoding unit 2000 decodes picture header informationin a bitstream received via a network. Information on the number, shape,position, and size of slices for each decoded rectangular region istransmitted to the slice constituting unit 2100, and other informationis transmitted to the slice decoding unit 2200.

The slice constituting unit 2100 selects positions of slices in responseto information on the number, shape, position, and size of slices foreach decoded rectangular region transmitted by the picture headerdecoding unit 2000, processes an overlapped portion of a rectangularregion, and constitutes slices. Processing of the overlapped portion ofthe rectangular region will be described below with reference to FIG.15.

The slice decoding unit 2200 decodes the image in units of slices byreferring to picture header information input from the picture headerdecoding unit 2000. For this purpose, the entropy decoding portion 2210decodes entropy of a bitstream, and the inverse-quantizing andinverse-transform portion 2220 performs inverse-quantization andinverse-transform on the entropy-decoded bitstream. Also, the imagerestoring portion 2230 compensates spatial/temporal prediction-codedinformation for output data of the inverse-quantizing andinverse-transform portion 2220 and restores the image. In this case, theimage restored in units of slices is added to a part of an image by theimage constituting unit 2300 in response to information input by theslice constituting unit 2100.

The image decoding apparatus having the above structure according to thepresent invention is used to decode an image coded by the image codingapparatus of FIG. 11. The operation of the image decoding apparatus isperformed in a reverse order to that of the image coding apparatus.However, principle characteristics for slice processing of the imagedecoding apparatus are the same as those of the image coding apparatus.Thus, for convenience, these operations will not be repeated here.

FIG. 14 shows a flowchart illustrating an image decoding methodperformed in the image decoding apparatus of FIG. 13. Referring to FIG.14, in step 2500, picture header information is decoded in a receivedbitstream. In this case, in step 2600, information on the number, shape,position, and size of slices for each decoded rectangular region istransmitted by the slice constituting unit 2100, and the sliceconstituting unit 2100 selects the positions of slices in response tothe input information, processes an overlapped portion of a rectangularregion, and constitutes slices. In step 2700, decoding is performed onthe constituted slices, and in step 2800, an image is constituted byreferred to the positions and sizes of the slices.

Here, slice decoding (step 2700) comprises decoding entropy of abitstream (step 2710), performing inverse-quantization andinverse-transform on the entropy-decoded data (step 2720), andcompensating spatial/temporal prediction-coded information for theinverse-transform performed data (step 2730).

FIGS. 15A and 15B illustrate a method of processing slices according tothe present invention. Referring to FIG. 15A, a part of an image 500includes two rectangular regions 502 and 503 and a background region504. The two rectangular regions 502 and 503 are overlapped with eachother. Also, the two rectangular regions 502 and 503 and the backgroundregion 504 are comprised of a plurality of independent slices.

The smaller rectangular region 502 of the two rectangular regions 502and 503 is completely included in a region of the larger rectangularregion 503. In this case, the larger rectangular region 503 represents aportion from which the smaller rectangular region 502 is excluded (aregion in which a hatched portion is excluded from the region of thelarger rectangular region 503) not to be overlapped on a region (ahatched portion) of the smaller rectangular region 502. In this case,the smaller rectangular region 502 includes a region which a user thinksmore important in the piece of image 500, i.e., a region of interest(ROI), and the larger rectangular region 503 is used as an interpolationregion placed between the ROI 501 and the background region 504. Here,the smaller rectangular region 502 including the ROI 501 and the largerrectangular region 503 used as the interpolation region representinformation on position and size of each rectangular region usingleft-upper position information 520 and 510 and right-lower positioninformation 521 and 511. As described above, the two rectangular regions502 and 503 and the background region 504 are comprised of a pluralityof independent slices and are coded and decoded in units of slices.Also, when the image shown in FIG. 15A is coded and decoded, preferably,the image is coded in the order of the smaller rectangular region 502,the larger rectangular region 503, and the background region 504, andregions can be discriminated by assigning the number of a region inaccordance with the order of a coded rectangular region. For example, asshown in FIG. 15A, the number of a region id=0 may be assigned to theforemost coded rectangular region 502, and the number of a region id=1may be assigned to the next coded rectangular region 503, and the numberof a region id=2 may be assigned to the background region 504.

Likewise, an interpolation region 503 is set between the ROI 501 and thebackground region 504 such that occurrence of a boundary between a ROIand a background region is effectively reduced.

Here, the plurality of slices constituting the smaller rectangularregion 502 can be independently decoded. Thus, when the decoding of thewhole image is unnecessary, only the slices constituting the smallerrectangular region including the ROI are decoded. In this case, an imagerestored from the smaller rectangular region 502 becomespicture-in-picture (PIP). The image restored from the larger rectangularregion 503 is added to the image restored from the smaller rectangularregion 502 and constitutes another larger PIP. Thus, PIP can be stepwiseconstituted as the number of rectangular regions in a part of an image,that is, from a small image to a larger image can be expressed.

In FIG. 15B, a part of an image 600 includes three rectangular regions603, 604, and 605, and a background region 606. The two rectangularregions 603 and 604 are not overlapped with each other and areoverlapped on another rectangular region 605. Also, the threerectangular regions 603, 604, and 605 and the background region 606 arecomprised of a plurality of independent slices.

Referring to FIG. 15B, each of the two rectangular regions 603 and 604includes different ROIs 601 and 602 which a user thinks more importantin the piece of image 600. Here, the smaller rectangular regions 603 and604 including the ROIs 601 and 602 and the larger rectangular region 605used as the interpolation region represent information on position andsize of each rectangular region using left-upper position information630, 620, and 610 and right-lower position information 631, 621, and611. As described above, the three rectangular regions 603, 604, and 605and the background region 606 are comprised of a plurality ofindependent slices and are coded and decoded in units of slices. Also,when the image shown in FIG. 15B is coded and decoded, preferably, theimage is coded in the order of the smaller rectangular regions 603 and604, the larger rectangular region 605, and the background region 606,and regions can be discriminated by assigning the number of a region inaccordance with the order of a coded rectangular region. For example, asshown in FIG. 15B, the number of a region id=0 may be assigned to theforemost coded rectangular region 603, and the number of a region id=1may be assigned to the next coded rectangular region 604, the number ofa region id=3 may be assigned to the next coded larger rectangularregion 605, and the number of a region id=4 may be assigned to thebackground region 606.

Likewise, an interpolation region 605 is set between the ROIs 601 and602 and the background region 606 such that occurrence of a boundarybetween a ROI and a background region is effectively reduced.

In this way, ROI coding or PIP coding can be effectively performed usinga slice structure in which inside of a rectangular region is a ROI andoutside of the rectangular region is a background region. In particular,as described above, several rectangular regions are overlapped with oneanother such that a boundary between regions is reduced and PIP havingvarious sizes can be supported. In addition, slices including ROIs arecoded to be more robust to errors such that a better subjective imagequality is obtained in a transmission environment with errors.

FIG. 16 shows comparison of a subjective image quality of a case where acoding method according to the present invention is applied to an actualimage, with a case where a conventional coding method is applied to anactual image. Referring to FIG. 16, a left column represents subjectiveimage quality using a conventional coding method, and a right columnrepresents subjective image quality using the coding method according tothe present invention, showing the results at a 20% packet loss ratio.In FIG. 16, it can be noted that image quality of a region of interest(ROI) is improved such that overall subjective image quality isimproved. Also, the ROI is more protected from errors such that overallsubjective image quality is improved.

FIG. 17 shows comparison of an objective image quality of a case where acoding method according to the present invention is applied to an actualimage, with a case where a conventional coding method is applied to anactual image. Referring to FIG. 17, a left column represents PSNR in thewhole image, and a right column represents PSNR in a region of interest(ROI). In FIG. 17, it can be noted that PSNR in the whole image and theROI using the coding method according to the present invention isimproved compared to the conventional coding method.

In addition, the present invention can be implemented as computerreadable codes on computer readable recording media. The computerreadable recording media include all kinds of recording apparatuses onwhich computer readable data is stored. The computer readable recordingmedia include ROMs, RAMs, CD-ROMs, magnetic tapes, floppy discs, andoptical data storage apparatuses, and further include carrier waves(i.e., transmission over the Internet). The computer readable recordingmedia are installed in a computer system that is connected to a network,and thus the computer readable codes can be stored and executed in adistributed mode.

As described above, according to the image coding method and apparatusconsidering human visual characteristics of the present invention, ascene is divided into a region of interest (ROI) and a backgroundregion, and modeling of image quality distribution is performed suchthat a difference in the image quality between the ROI and thebackground region is not rapid, thereby improving image quality. Inaddition, ROI coding can be effectively performed using rectangularregions, each region being independently coded and decoded in units ofslices. In particular, rectangular regions are overlapped with oneanother such that occurrence of a boundary between the ROI and thebackground region is effectively prevented, thereby improving subjectiveimage quality. Further, there is no need of iteratively readjusting thequantization parameter for each region, so as to meet given amount ofbit, and thus a large amount of calculation is not needed.

While this invention has been particularly shown and described withreference to preferred embodiments thereof, it will be understood bythose skilled in the art that various changes in form and details may bemade therein without departing from the spirit and scope of theinvention as defined by the appended claims.

1. An image decoding apparatus comprising: a picture header decodingunit which decodes a picture header in a bitstream; a slice constitutingunit which constitutes slices using information on positions and sizesof slices among the picture header information; a slice decoding unitwhich decodes an image in units of slices by referring to the pictureheader; and an image constituting unit which constitutes the image inunits of slices restored by the slice decoding unit as a part of animage by referring to the information on positions and sizes of theslices obtained by the slice constituting unit.
 2. The apparatus asclaimed in claim 1, wherein the slice constituting unit sets an innerrectangular region including a region of interest, an outer rectangularregion including the inner rectangular region, a background regionincluding a region excluding the outer rectangular region by referringto the picture header, and divides the inner rectangular region, aregion between the inner rectangular region and the outer rectangularregion, and the background region into at least one of independentslices.
 3. The apparatus as claimed in claim 2, wherein the outerrectangular region is an interpolation region interpolated so that aboundary between the inner rectangular region and the background regiondose not appear.
 4. The apparatus as claimed in claim 2, wherein theslice decoding unit constitutes picture-in-picture (PIP) having a smallsize using slices constituting the inner rectangular region, andconstitutes another PIP using the slices constituting the innerrectangular region and slices constituting the region between the innerrectangular region and the outer rectangular region.
 5. The apparatus asclaimed in claim 1, wherein the slice constituting unit sets a pluralityof inner rectangular regions including a plurality of regions ofinterest, an outer rectangular region including the plurality of innerrectangular regions, a background region including a region excludingthe outer rectangular region by referring to the picture header, anddivides the plurality of inner rectangular regions, the region betweenthe plurality of inner rectangular regions and the outer rectangularregion, and the background region into at least one of independentslices.
 6. The apparatus as claimed in claim 5, wherein the outerrectangular region is an interpolation region interpolated so that aboundary between the plurality of inner rectangular regions and thebackground region does not appear.
 7. The apparatus as claimed in claim1, wherein the slice decoding unit comprises: an entropy decodingportion which decodes entropy of the bitstream; an inverse-quantizingand inverse-transform portion which performs inverse-quantization andinverse-transform on information decoded by the entropy decodingportion; and an image restoring portion which spatial/temporalprediction-compensates the information obtained by theinverse-quantizing and inverse-transform portion and restores the imagein units of slices.
 8. An image decoding apparatus comprising: a pictureheader decoding unit which decodes a picture header in a bitstream; aslice constituting unit which constitutes slices using information onpositions and sizes of slices among the picture header information; aslice decoding unit which decodes an image in units of slices byreferring to the picture header; and an image constituting unit whichconstitutes the image in units of slices restored by the slice decodingunit as a part of an image by referring to the information on positionsand sizes of the slices obtained by the slice constituting unit, whereinthe slice constituting unit sets an inner rectangular region including aregion of interest, an outer rectangular region including the innerrectangular region, a background region including a region excluding theouter rectangular region by referring to the picture header, and dividesthe inner rectangular region, a region between the inner rectangularregion and the outer rectangular region, and the background region intoat least one of independent slices.